SF  298  MASTER  COPY 


KEEP  THIS'  COPY  FOR  REPRODUCTION  PURPOSES 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
0MB  NO.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comment  regarding  this  burden  estimates  or  any  other  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services.  Directorate  for  information  Operations  and  Reports,  1215  Jefferson 
Davis  Highway.  Suite  1204,  Arlington.  VA  22202-4302.  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 


1.  AGENCY  USE  ONLY  (Leave  blank) 


4.  TITLE  AND  SUBTITLE 


2.  REPORT  DATE 

10/8/96 


3.  REPORT  TYPE  AND  DATES  COVERED 

Final  Report  9/92  -  8/96 


5.  FUNDING  NUMBERS 


Deformable  Topological  Templates  for  Image  Analysis 


DAAL03-92-G-0322 


6,  AUTHOR(S) 

Yali  AMIT 


7.  PERFORMING  ORGANIZATION  NAMES(S)  AND  ADDRESS(ES) 

Department  of  Statistics 
Tbe  University  of  Chicago 
573^  University  Avenue 
Chicago,  Illinois  60637 


9.  SPONSORING  /  MONITORING  AGENCY  NAME{S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Office 
P.O.Box  12211 

Re.search  Triangle  Park,  NC  27709-2211 


11,  SUPPLEMENTARY  NOTES 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSORING  /  MONITORING 
AGENCY  REPORT  NUMBER 


The  yiew.s,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and  should  not  be  construed  as 
an  official  Department  of  the  Army  position,  policy  or  decision,  unless  so  designated  by  other  documentation. 


12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 


Approved  for  public  release;  distribution  unlimited. 


13.  ABSTRACT  (Maximum  200  words) 


19961025  001 


The  main  achievements  of  this  proposal  has  been  the  introduction  of  a  new  family  of 
global  but  sparse  image  features  consisting  of  geometric  arrangements  of  local  responses, 
These  features  can  serve  as  building  blocks  for  template  models,  allowing  for  explicit 
modeling  of  the  variability  of  the  object  family,  and  efficient  computation  of  the 
template  match.  These  features  can  also  be  accessed  through  decision  trees  to  provide 
very  accurate  and  efficient  shape  classifiers. 


14.  SUBJECT  TERMS 


15.  NUMBER  IF  PAGES 


DTIC  QtTAim  mSPBCTED  S 


16.  PRICE  CODE 


17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION  20.  LIMITATION  OF  ABSTRACT 
OR  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

UNCLASSIRED  UNCLASSIFIED  UNCLASSIFIED  UL 


.MSN  7540-01-280-5500 


Enclosure  1 


Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Sid.  239-18 
298-102 


FINAL  REPORT:  August  1,  1992  -  August  30,  1996 


DEFORMABLE  TOPOLOGICAL  TEMPLATES  FOR  IMAGE  ANALYSIS 


ARO  Proposal  no.  30167-MA 
Grant  no.  -  DAAL03-92-G-0322 


Yali  Amit 

Department  of  Statistics 
University  of  Chicago 


October  8,  1996 


1 


FINAL  PROGRESS  REPORT 


1  Publications 


Am.it,  Y.  (1992a),  Introduction  to  gibbs  distributions  and  large  deviations  of  their  empirical 
distributions.  Technical  report.  Department  of  Statistics,  University  of  Chicago, 

Amit,  Y.  (19926),  On  the  equivalence  of  functions  on  the  unit  square  modulo  composition. 
Technical  Report  338,  Department  of  Statistics,  University  of  Chicago, 

Amit,  Y.  &  Miller,  M,  (1992a),  ‘Large  deviations  for  coding  markov  chains  and  gibbs  random 
fields’,  IEEE  Transactions  on  Information  Theory  39(1),  109-119, 

Amit,  Y.  &  Miller,  M,  (19926),  ‘Large  deviations  for  the  asymptotics  of  ziv-lempel  codes  for 
2-d  gibbs  fields’,  IEEE  Transactions  on  Information  Theory  38(4:),  1271-1276. 

Amit,  Y.,  G.  U.  M.  M.  (1992c),  Ergodic  properties  of  jump  diffusion  processes.  Technical 
Report  361,  Department  of  Statistics,  University  of  Chicago. 

Amit,  Y.  &  Manbeck,  K.  M.  (1993),  ‘Deformable  template  models  for  emission  tomography’, 
IEEE  Transactions  on  Medical  Imaging  12(2),  260-269. 

Amit,  Y.  &  McCullagh,  R  (1993),  Extremal  convex  sets  and  the  three  corner  theorem. 
Technical  Report  372,  Department  of  Statistics,  University  of  Chicago. 

Miller,  M.,  Christensen,  G.,  Amit,  Y.  k  Grenander,  U.  (1993),  ‘A  mathematical  textbook  of 
deformable  neuro-anatomies’,  Proc.  of  the  National  Academy  of  Science  R90,  11944- 
11948. 

Shun,  Z.  (1994),  Some  results  connected  with  random  effects  in  logistic  models.  Technical 
report  398,  Department  of  Statistics,  University  of  Chicago. 

Amit,  Y.  (1994),  Large  deviations  and  the  rate  distortion  theorem  for  gibbs  distributions,  in 
‘Proceedings  of  the  IT/STAT  Joint  lEEE/IMS  workshop’. 


2 


Amit,  Y.  &  Geman,  D.  (1994),  Randomized  inquiries  about  shape;  an  application  to  hand¬ 
written  digit  recognition,  Technical  Report  401,  Department  of  Statistics,  University  of 
Chicago. 

Amit,  Y.  (1994),  ‘Discussion  of  representation  of  complex  systems  by  ulf  grenander  and 
michael  miller’,  Journal  of  the  Royal  Statistical  Society  B. 

Collins,  L.,  Pluzhnikov,  A.  &  Stein,  M.  L.  (1994),  Improvement  of  inter-event  distance  tests  of 
randomness  in  spatial  point  processes.  Technical  Report  384,  Department  of  Statistics, 
University  of  Chicago. 

Amit,  Y.,  Geman,  D.  &  Wilder,  K.  (1995),  Recognizing  shapes  from  simple  queries  about 
geometry.  Technical  report.  Department  of  Statistics,  University  of  Chicago. 

Ambrosius,  W.  (1995),  Deformable  templates  and  image  compression,  Doctoral  Dissertation, 
UNiversity  of  Chicago. 

Amit,  Y.  (1995),  ‘Graphical  shape  templates  for  automatic  anatomy  detection:  application 
to  mri  brain  scans’,  to  appear  IEEE  Trans.  Medical  Imaging. 

FLoresroux,  E.  M.  &  Stein,  M.  L.  (1996),  ‘A  new  method  of  edge  correction  for  estimating  the 
nearest  neighbor  distribution’.  Journal  of  Statistical  Planning  and  Inference  50,  353- 
371. 

Amit,  Y.  (1996),  ‘Convergence  properties  of  the  gibbs  sampler  for  perturbations  of  gaussians’. 
Annals  of  Statistics  24(1),  122-140. 

Amit,  Y.  &  Geman,  D.  (1996),  Shape  quantization  and  recognition  with  randomized  trees. 
Technical  report.  Department  of  Statistics,  University  of  Chicago. 

Amit,  Y.  Sz  Kong,  A.  (1996),  ‘Graphical  templates  for  model  registration’,  IEEE  PAMI 
18,  225-236. 

Geman,  D.,  Amit,  Y.  &  Wilder,  K.  (1996),  Joint  induction  of  shape  features  and  tree  classi¬ 
fiers,  Technical  report.  Department  of  SMathematics  and  Statistics,  University  of  Mas¬ 
sachusetts,  Amherst. 


3 


2  Scientific  Personnel 


Undergraduate  (summer  support): 

Thomas  Scanlon. 

Graduate  students  (summer  support): 

Zhenming  Shun,  (Shun  1994). 

Ernesto  Floresroux,  (Floresroux  &  Stein  1996) 

Linda  Collins,  Anna  Pluzhnikov,  (Collins  et  al.  1994) 

Graduate  student:  Walter  Ambrosius. 

Ph.D.  December  1995. 

Title:  Deformable  Templates  and  Image  Compression. 

Graduate  student:  Steve  Wang. 

Research  topic:  Automatic  zipcode  identification 
Graduate  student:  Jiayu  Li. 

Research  topic:  Graphical  templates  for  medical  image  analysis. 


3  Summary 

The  initial  goals  of  this  proposal  were  the  following; 

1.  Apply  2-dimensional  elastic  deformable  template  models  to  concrete  image  analysis 
problems  such  as  tomography  reconstruction,  movement  compensation  and  image  com¬ 
pression. 

2.  Analyze  these  models  on  the  theoretical  level,  including  a  rigorous  formulation  of  the 
relevant  topological  issues. 

3.  Use  these  results  to  obtain  a  more  systematic  mechanism  for  generating  templates,  for 
determining  the  appropriate  class  of  transformations  and  the  appropriate  priors. 

4.  Try  to  apply  deformable  template  methods  to  problems  in  object  recognition. 

The  application  of  the  elastic  deformation  models  to  tomography  was  attempted  in  the 
context  of  tumor  detection.  Simple  models  for  the  tumor  and  the  scalp  were  formulated 
together  with  a  low  dimensional  family  of  allowable  linear  and  non-linear  perturbations. 
Instead  of  reconstructing  the  image  using  standard  techniques  and  then  searching  for  the 
tumor  in  the  reconstruction,  the  deformation  parameters  of  the  model  were  obtained  directly 
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from  the  count  data,  using  the  forward  model  of  SPECT  data  generation.  The  actual 
optimization  was  carried  out  on  a  weighted  least  squares  approximation  to  the  Poisson  model. 
This  is  described  in  progress  report  no.  1  (September,  1992)  and  in  (Amit  k  Manbeck  1993). 

Deformable  template  methods  were  applied  to  the  compression  of  images  of  a  given 
medical  object.  A  template  image  was  determined,  and  image  matching  techniques  were 
used  to  warp  the  template  into  a  data  image.  The  warping  is  described  with  a  small  number 
of  parameters.  The  residual  image  is  very  smooth  and  contains  very  little  structure,  and  is 
compressed  through  tree  structured  vector  quantization  techniques,  which  are  trained  using 
several  sample  images.  This  method  is  described  in  progress  reports  no.  4  (February,  1995) 
and  no.  5.  (February,  1996)  and  in  (Ambrosius  1995). 

The  analysis  of  the  topological  aspects  of  the  2-d  elastic  deformation  models  is  descried 
in  progress  report  no.  2  (January,  1993)  and  in  (Amit  19926).  This  analysis  led  to  the 
development  of  graphical  template  models  which  solve  many  of  the  issues  raised  in  item  3 
above.  The  graphical  models  are  descriptions  of  geometric  arrangements  of  landmarks.  The 
landmarks  are  defined  through  simple  local  operators,  various  linear  or  non-linear  filters. 
The  models  are  invariant  to  a  large  range  of  scales,  translations  and  certain  deformations. 
The  constraints  imposed  by  the  geometric  arrangement  ensure  that  the  chance  of  a  random 
occurrence  is  very  low,  and  typically  a  good  arrangement  will  be  found  only  at  the  correct 
locations  on  the  object  which  is  being  modeled.  The  invariances  ensure  robustness  to  the 
natural  variability  in  shapes  of  given  anatomies.  These  models  are  described  in  progress 
reports  no.  3  (March,  1994),  and  no.  4  (February,  1995),  as  well  as  in  (Amit  1994), (Amit 
k  Kong  1996)  and  (Amit  1995).  In  the  latter  publication  an  extensive  test  of  the  algorithm 
on  204  axial  MRI  brain  images  from  68  patients  is  reported. 

Graphical  templates  allow  for  explicit  modeling  of  the  templates,  and  for  more  appro¬ 
priate  definition  of  the  relevant  transformations.  They  are  very  efficient  computationally, 
and  avoid  the  issues  of  local  minima  encountered  in  the  relaxation  methods  used  to  solve 
the  elastic  models,  moreover  no  initialization  is  required.  Indeed  the  graphical  template 
methods  can  serve  as  initialization  to  elastic  deformation  models.  This  was  heavily  used  in 
the  compression  algorithms  described  in  (Ambrosius  1995). 

Initially  it  was  proposed  to  use  templates  for  object  recognition.  Various  drawbacks  of 
this  approach  were  soon  encountered.  Most  important  is  the  computational  issue  of  trying 
all  templates  on  a  given  object.  This  becomes  a  serious  concern  in  problems  with  hundreds 
of  object  types.  The  modeling  issue,  generating  templates  for  each  object  is  also  a  consid¬ 
erable  problem.  However  it  was  noted  that  the  basic  ingredients  of  the  graphical  template 
models,  e.g.  geometric  arrangements  of  landmarks  representing  local  topographic  codes. 
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could  serve  as  of  universal  features  for  object  recognition.  Since  geometric  arrangements 
of  local  responses  can  be  described  through  labeled  graphs  there  is  a  natural  partial  order¬ 
ing  of  these  arrangements,  involving  more  and  more  complex  arrangements.  Decision  trees 
are  particularly  suitable  for  accessing  this  partially  ordered  set  of  features.  They  can  be 
used  simultaneously  to  identify  the  informative  features  for  a  certain  classification  problem 
and  as  classifiers.  Multiple  and  randomized  trees  are  used  to  overcome  estimation  problems 
associated  with  small  data  sets  and  achieve  a  high  degree  of  robustness  of  the  classifier. 

The  application  of  these  ideas  to  shape  recognition  is  described  in  progress  reports  no.  4 
and  no.  5  and  in  (Amit  &  Geman  1994),  (Amitet  al.  1995),  (Geman  et  al.  1996),  and  (Amit 
k  Geman  1996).  Experiments  with  hundreds  of  shape  classes  corresponding  to  perturbed 
versions  of  293  LaTeX  symbols  show  the  scalability  of  this  method  to  very  large  dimensional 
problems.  Furthermore  possible  connection  with  functions  of  the  visual  cortex  have  been 
explored. 

The  main  achievement  of  the  research  over  the  past  four  years  has  been  the  use  of  sparse 
image  features  in  the  form  of  global  geometric  arrangements  of  local  responses,  both  for  reg¬ 
istration  and  for  recognition.  The  algorithms  are  very  efficient  and  accurate.  In  recognition 
problems,  state  of  the  art  classification  rates  were  achieved  on  the  NIST  handwritten  digit 
database. 

The  current  aim  of  research  is  to  achieve  full  integration  of  the  registration  and  the 
recognition  methods.  This  needs  to  be  done  at  several  levels. 

1.  Modeling:  use  decision  tree  techniques  to  derive  the  graph  models  automatically  from 
learning  data,  instead  of  one  model,  many  can  be  derived  thus  providing  more  robust 
registrations. 

2.  Focusing;  use  the  registration  methods  to  identify  important  sub-regions  in  the  image, 
which  can  be  further  analyzed  using  classification  trees.  This  should  yield  new  and 
efficient  object  finding  algorithms,  for  example  face  finding  algorithms. 

3.  Final  testing  through  registration:  template  registration  can  be  used  for  disambigua¬ 
tion  between  a  small  number  of  candidate  classes  chosen  by  the  classification  algorithm. 
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